Reconstructing 3D Models from Movies with Canonical Cinematography Settings

Reconstructing the 3D scene depicted by a video clip is an active field of research. For this task, we need two or more overlapping images of the same situation taken from different angles. After that, we can recover the depth of points that appear in both images. Characteristics of modern movies makes it difficult to reconstruct 3D scenes from them. They often utilize cameras that with more than 45◦ difference in orientation, large differences in focol length, and directly cut from one camera to another. Figure 1 shows an example of a canonical cinematography setting, known as the 180-degree rule. The bottom 3 camera positions are commonly used, and the top position is disallowed [4]. The drastic changes in cameras result in small regions of overlap, creating difficulties for finding points that appear in both images. However, cinematographic conventions are not meant to confuse the audience. Rather, they facilitate comprehension of the movie, arguably because they cater to human cognition [5]. Hence, a reasonable hypothesis is that knowledge of cinematic conventions, more specifically knowledge of camera poses and positions, can help computational systems understand movie as well. This paper investigates 3D reconstruction under the adverse condition of modern cinematography. Small overlaps between images lead to many wrongly matched points. Therefore, the key to finding correct matches is to increase the signal-to-noise ratio. In this paper, we attempt to employ the 180-degree rule and other conventions to filter out wrong matches. We find that knowledge on camera positions is beneficial for 3D reconstruction, but still insufficient for some difficult situations. 2. Related Work